This paper is in the following e-collection/theme issue:

Reviews in Digital Mental Health (223) e-Mental Health and Cyberpsychology (1372) Clinical Communication, Electronic Consultation and Telehealth (537) Posttraumatic Stress Disorder (PTSD) (105) Anxiety and Stress Disorders (989) Depression and Mood Disorders; Suicide Prevention (1260) Telepsychiatry (143)

Published on in Vol 10 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/44790, first published .
Psychiatric Treatment Conducted via Telemedicine Versus In-Person Modality in Posttraumatic Stress Disorder, Mood Disorders, and Anxiety Disorders: Systematic Review and Meta-Analysis

Psychiatric Treatment Conducted via Telemedicine Versus In-Person Modality in Posttraumatic Stress Disorder, Mood Disorders, and Anxiety Disorders: Systematic Review and Meta-Analysis

Psychiatric Treatment Conducted via Telemedicine Versus In-Person Modality in Posttraumatic Stress Disorder, Mood Disorders, and Anxiety Disorders: Systematic Review and Meta-Analysis

Authors of this article:

Ali Abbas Shaker1, 2 Author Orcid Image ;   Stephen F Austin1, 3 Author Orcid Image ;   Ole Jakob Storebø1, 3 Author Orcid Image ;   Julie Perrine Schaug1 Author Orcid Image ;   Alaa Ayad1 Author Orcid Image ;   John Aasted Sørensen4 Author Orcid Image ;   Kristine Tarp5, 6 Author Orcid Image ;   Henrik Bechmann4 Author Orcid Image ;   Erik Simonsen2, 7 Author Orcid Image
Article Authors Cited by (3) Tweetations (9) Metrics

    Review

    1Psychiatric Department, Region Zealand Psychiatry, Psychiatric Research Unit, Slagelse, Denmark

    2Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark

    3Department of Psychology, University of Southern Denmark Faculty of Health Sciences, Odense, Denmark

    4Department of Engineering Technology and Didactics, Research unit: AI, Mathematics and Software, Technical University of Denmark, Ballerup, Denmark

    5Research Unit for Digital Psychiatry, Mental Health Services in the Region of Southern Denmark, Odense, Denmark

    6Department of Clinical Research, University of Southern Denmark, Odense, Denmark

    7Mental Health Services East, Copenhagen University Hospital – Psychiatry Region Zealand, Roskilde, Denmark

    Corresponding Author:

    Ali Abbas Shaker, BEng, MD

    Psychiatric Department, Region Zealand Psychiatry, Psychiatric Research Unit

    Fælledvej 6

    Slagelse, 4200

    Denmark

    Phone: 45 58536070

    Email: ashak@regionsjaelland.dk


    Related ArticleThis is a corrected version. See correction statement in: https://mental.jmir.org/2023/1/e52269

    Background: Telemedicine has played a vital role in providing psychiatric treatment to patients during the rapid transition of services during the COVID-19 pandemic. Furthermore, the use of telemedicine is expected to expand within the psychiatric field. The efficacy of telemedicine is well described in scientific literature. However, there is a need for a comprehensive quantitative review that analyzes and considers the different clinical outcomes and psychiatric diagnoses.

    Objective: This paper aimed to assess whether individual psychiatric outpatient treatment for posttraumatic stress disorder, mood disorders, and anxiety disorders in adults using telemedicine is equivalent to in-person treatment.

    Methods: A systematic search of randomized controlled trials was conducted using recognized databases for this review. Overall, 4 outcomes were assessed: treatment efficacy, levels of patient satisfaction, working alliance, and attrition rate. The inverse-variance method was used to summarize the effect size for each outcome.

    Results: A total of 7414 records were identified, and 20 trials were included in the systematic review and meta-analysis. The trials included posttraumatic stress disorder (9 trials), depressive disorder (6 trials), a mix of different disorders (4 trials), and general anxiety disorder (1 trial). Overall, the analyses yielded evidence that telemedicine is comparable with in-person treatment regarding treatment efficacy (standardized mean difference −0.01, 95% CI −0.12 to 0.09; P=.84; I2=19%, 17 trials, n=1814), patient satisfaction mean difference (−0.66, 95% CI −1.60 to 0.28; P=.17; I2=44%, 6 trials, n=591), and attrition rates (risk ratio 1.07, 95% CI 0.94-1.21; P=.32; I2=0%, 20 trials, n=2804). The results also indicated that the working alliance between telemedicine and in-person modalities was comparable, but the heterogeneity was substantial to considerable (mean difference 0.95, 95% CI −0.47 to 2.38; P=.19; I2=75%, 6 trials, n=539).

    Conclusions: This meta-analysis provided new knowledge on individual telemedicine interventions that were considered equivalent to in-person treatment regarding efficacy, patient satisfaction, working alliance, and attrition rates across diagnoses. The certainty of the evidence regarding efficacy was rated as moderate. Furthermore, high-quality randomized controlled trials are needed to strengthen the evidence base for treatment provided via telemedicine in psychiatry, particularly for personality disorders and a range of anxiety disorders where there is a lack of studies. Individual patient data meta-analysis is suggested for future studies to personalize telemedicine.

    Trial Registration: PROSPERO International Prospective Register of Systematic Reviews CRD42021256357; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=256357

    JMIR Ment Health 2023;10:e44790

    doi:10.2196/44790

    Keywords

    telemedicine; telepsychiatry; video consultation; mobile health; mHealth; eHealth; COVID-19; synchronous technology; anxiety; psychiatry; patient satisfaction; depression; posttraumatic stress disorder; PTSD 


    Background

    During the last 2 decades, there has been increasing interest in and publication of research studies addressing the effect of telemedicine on psychiatric patients [1-5]. Research has highlighted several potential advantages of using telemedicine in mental health services. Some of the most apparent advantages of integrating telemedicine in mental health services are enabling clinicians to reach out to patients living in rural areas and patients with mental health difficulties who find it challenging to attend treatment in person [1,2]. However, several studies have also reported challenges regarding the use of telemedicine for mental health services, including concerns about establishing a good patient-therapist alliance and the underuse of telemedicine by clinicians in resource-constrained clinics [3-5].

    The experience of COVID-19 has placed an increased focus on the provision of interventions using telemedicine. This unique world situation, coupled with continual advances in technology, means that a regular synthesis of evidence for psychiatric interventions using telemedicine is warranted [6-10].

    In recent years, several meta-analyses have compared the efficacy of psychiatric treatment provided using telemedicine with in-person treatment [11-13]. Drago et al [11] reviewed the evidence of psychiatric counseling (but not specific psychiatric or psychotherapeutic interventions) using telemedicine compared with in-person treatment. They included 24 randomized controlled trials (RCTs) primarily for posttraumatic stress disorder (PTSD) and major depression and found no difference in treatment effects between the 2 modalities. Their review did not examine the satisfaction, alliance, or attrition rates between the 2 modes of treatment. Batastini et al [12] conducted a large meta-analysis with broad inclusion criteria and combined data from a variety of study designs (RCTs and within subjects), reported outcomes (observer rated and self-report), and treatment format (individual and group). Their analysis included 43 studies and found that treatment effects were largely comparable between telemedicine and in-person modalities; however, they did not examine satisfaction, alliance, or attrition rates between the 2 modes of treatment. Giovanetti et al [13] focused exclusively on comparing psychotherapeutic interventions for depression. They included 11 RCTs and found that telemedicine-based psychotherapy had comparable efficacy with in-person psychotherapy [13]. They also found no differences in the attrition rates between the 2 modalities in patients diagnosed with depression.

    Objective

    This meta-analysis builds on the results of previous reviews by addressing some of the deficiencies of earlier meta-analyses and providing a comprehensive and updated overview of the evidence for telemedicine in psychiatric settings. Thus, the primary aim of this systematic review and meta-analysis is to examine whether individual psychiatric outpatient interventions for adults using telemedicine are equivalent to the in-person format regarding treatment efficacy. As part of this comprehensive meta-analysis comparing the treatment effects between telemedicine and in person, we examined different diagnostic disorders and analyzed a range of moderators. Second, the meta-analysis addressed several gaps in current scientific research using standard and valid measures to examine the satisfaction, working alliance, and attrition rates between telemedicine and in-person modalities across a range of psychiatric diagnoses.


    Overview

    The methods section of this systematic review and meta-analysis is described in a published peer-reviewed protocol [14]. This systematic review was conducted according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [15]. The PRISMA checklist can be found on the web (Multimedia Appendix 1).

    Registration and Protocol

    This systematic review was registered in PROSPERO (CRD42021256357). A peer-reviewed protocol has been published for this study. First, a post hoc analysis was conducted to evaluate attrition outcomes based on diagnosis (PTSD and Depression). Second, subgroup analysis for the moderators, “settings,” and “vulnerable populations” were poorly described in the included studies, and subgroup analysis for these moderators was therefore not applicable. Third, the title names have been adjusted. Apart from the listed amendments, no significant amendments were made compared with the published protocol.

    Inclusion Criteria

    The eligibility criteria were based and restricted on the type of study, population, intervention, comparator, and outcomes of the studies.

    Types of Studies

    RCTs were considered.

    Types of Participants

    The participants were (1) adults (aged >18 years), (2) receiving individual psychiatric outpatient treatment, and (3) diagnosed with PTSD, mood disorders, anxiety, or personality disorders according to both the American Psychiatric Association’s Diagnostic and Statistical Manual of Mental Disorders III-V and the World Health Organization’s International Statistical Classification of Diseases 9 or 10. Participants with comorbid diagnoses were also included, with the exception of those diagnoses covered in the exclusion criteria.

    Types of Interventions

    Individual treatment through synchronous real-time video consultations in outpatient settings. Treatment was defined as an intervention that involved psychotherapy, pharmacological treatment, or psychoeducation.

    Types of Comparators or Controls

    The comparator was individual treatment in person with the same active treatment that the intervention group (telemedicine) received.

    Types of Outcomes

    The primary outcome was studies that assessed psychopathology (efficacy) after using a mental health service. The secondary outcomes of interest were (1) patient satisfaction, (2) working alliance, and (3) attrition rate.

    Exclusion Criteria

    The following criteria were considered the reason for exclusion:

    • Participant aged <18 years
    • Group therapy
    • Different psychotherapeutic (treatments) approaches used in the telemedicine and in-person modalities
    • Trials involving populations primarily treating psychotic disorders, mental retardation, bipolar disorder, alcohol abuse, and substance use disorders
    • Trials using asynchronous communication systems as an intervention (eg, emails and static websites without video function) and telephones with only audio function as an intervention

    Information Sources and Search Strategy

    The first step in the systematic review was a comprehensive search in electronic databases. The database search strategy was developed using input from the project team. A search was conducted for studies published between 1967 and October 2022.

    The following databases were used: MEDLINE (PubMed interface, 1986 onward), APA PsycINFO (Ovid interface, 1967 onward), Embase (Ovid interface, 1974 onward), Web of Science (Clarivate interface, 2001 onward), and CINAHL (EBSCOhost interface, 1981 onward).

    Medical Subject Headings (MeSH) and text words related to the search terms “psychiatry” and “telemedicine” were used to develop the search string in MEDLINE.

    Examples of MeSH and text words related to the term “psychiatry” included: (“Psychiatry”[MeSH Terms] or “Mental Disorders”[MeSH Terms] or “Mental Health Services”[MeSH Terms]) and (“mental health counseling”[Title/Abstract] or “mental health care”[Title/Abstract] or “psychiatric home care”[Title/Abstract] or “psychiatric outpatient*”[Title/Abstract]).

    Examples of MeSH and text words related to the term “telemedicine” included: (“Telemedicine”[MeSH Terms] or “Videoconferencing”[MeSH Terms] or “Remote Consultation”[MeSH Terms] and “telecare”[Title/Abstract] or “teleconsultation*”[Title/Abstract] or “telemedic*”[Title/Abstract] or “telepsychiatr*”) Both search terms “psychiatry” and “telemedicine” were combined with (AND).

    Specific syntax and subject headings were subsequently adapted individually to the different databases.

    No language or date restrictions were implemented in the search process. Owing to the preliminary search’s unmanageable results (>20.000 hits), the highly sensitive search strategy filters of Cochrane identifying randomized trials were applied. Unpublished studies were not sought.

    The second step in the search strategy was a manual literature search to identify additional primary studies for systematic review. The third step involved scanning the reference lists of the included studies or relevant reviews identified in the first and second steps, respectively.

    Data Management

    Records from the literature search were exported to the reference manager Endnote X9 (Clarivate Analytics) [16]. From Endnote, records were exported to Covidence (Veritas Health Innovation), a web app tool that facilitates collaboration among review team members during the study selection and data extraction process [17]. Data extracted in Covidence were exported to RevMan 5.4 (Cochrane Collaboration) for data analysis [18].

    Selection Process

    AAS and AA were responsible for the selection process. The 2 authors independently screened the titles and abstracts of the records in Covidence to identify potentially eligible records. The second step involved screening full-text reports to assess whether the reports met the eligibility criteria. Three authors (pairwise) were responsible for the second step (AAS, SFA, or JPS). Disagreements in the full-text screening process were resolved through discussions between the authors. A fourth reviewer, OJS, was consulted in case of continued disagreement despite discussion. The selection process was documented in the PRISMA flow diagram, including reasons for exclusion. Interrater reliability was measured using Cohen κ coefficient for the title and abstract screening and full-text review processes.

    Data Collection Process

    AAS, SFA, and JPS were responsible for the data collection process. Data extraction was performed independently by 2 authors using a standardized electronic data extraction form in Covidence. The data extraction form was pilot tested on 5 reports, and the reviewers met and discussed the form before starting the review. Disagreements in the data collection process were resolved through discussions between the authors. A third reviewer (OJS) was consulted when disagreements could not be resolved between the independent authors. If multiple reports of the same study were encountered, data from all reports were extracted into a single data collection form in Covidence [19]. Missing data were obtained by contacting and requesting these data from the study authors.

    Data Items

    We extracted the following data items for each study: (1) study characteristics (authors, author contact details, aim of the study, trial design, location, trial size, sample size calculation, year of publication, and country); (2) population characteristics (remote or rural area or urban, country, diagnosis or condition, mean age, and sex); (3) intervention or control (internet connection speed, bandwidth, therapy type, number of consultation sessions, and duration of consultation); and (4) clinical outcome (assessment tools, psychopathology [efficacy outcome], patient satisfaction, working alliance, and attrition rate). When reported in the studies, we collected data from the intention-to-treat analysis; otherwise, we collected data from the per-protocol analysis.

    Outcomes and Prioritization

    The primary outcome was efficacy, as assessed by clinician or patient-rated scales. As we expected that different assessment tools had been used for measuring the primary outcome, we prioritized clinician-rated scales over patient-rated scales, should both be available.

    The secondary outcomes were (1) patient satisfaction, (2) working alliance, and (3) attrition rate. The patient satisfaction measure was restricted to the Client Satisfaction Questionnaire-8 (CSQ-8) [20], and the working alliance was restricted to the Working Alliance Inventory-Client version (WAI-C) [21]. The attrition rate was defined as the proportion of individuals who withdrew after being randomized to a modality to the total number of participants randomized to a modality.

    Risk of Bias in Individual Studies

    Authors AAS, SFA, and JPS performed (pairwise) the risk of bias (quality) assessment for the primary outcome (treatment efficacy) in each individual study, using the revised Cochrane risk-of-bias tool for randomized trials (RoB 2) [19]. The bias domains assessed included (1) bias arising from the randomization process, (2) bias due to deviations from intended interventions, (3) bias due to missing outcome data, (4) bias in the measurement of the outcome, and (5) bias in the selection of the reported result. The overall risk of bias for each study was marked as (1) “low risk of bias” if all domains were judged to be at low risk of bias, (2) “some concerns” if at least one domain was judged to raise some concerns but not to be at high risk of bias for any domain, or (3) “high risk of bias” if any domain was judged to be at a high risk of bias. Disagreements between the mentioned researchers regarding the risk of bias were resolved through consensus or by a third researcher (OJS). The Covidence tool was used to assess the risk of bias.

    Data Synthesis (Statistical Methods)

    The general strategy for data synthesis was to perform a quantitative synthesis (meta-analysis). Heterogeneity (I²) values were judged as follows: 0%-40% may represent little heterogeneity, 30%-60% may represent moderate heterogeneity, 50%-90% may represent substantial heterogeneity, and 75%-100% may represent considerable heterogeneity. Heterogeneity, which is the percentage of variation across studies owing to heterogeneity rather than chance, was evaluated for clinical, methodological, and statistical heterogeneity [19].

    Quantitative Synthesis

    We expected clinical and methodological heterogeneity in the pooled studies; therefore, we applied a random effects model to obtain the overall effect size estimate. The inverse-variance method was used to perform the meta-analysis. Larger studies with less variance were given more weight in the meta-analysis owing to more precise effect size estimates than smaller studies.

    Continuous Outcome Measures

    The standardized mean difference (SMD) effect size was calculated for the primary outcome using the Hedges g formula. Different assessment tools were used to calculate the effect size of the primary outcome in each study. Therefore, SMD was statistically suitable for estimating the effect size for each study. Forest plots were used to present study-specific effect sizes and overall effect sizes, including 95% CIs. Furthermore, we calculated the I² statistic to quantify heterogeneity and the χ² statistic to test for heterogeneity (P≤.10 significance level).

    For the secondary outcomes—patient satisfaction and working alliance—the mean difference (MD) effect size was calculated as these secondary outcomes were assessed using a single standardized tool (CSQ-8 and WAI-C). Therefore, standardization was not needed to calculate the effect size across studies. The same statistical approach used for the primary outcome was applied to the secondary outcomes of patient satisfaction and working alliance.

    Postintervention data (sample size, mean, and SD) for each treatment modality (in person and telemedicine) were used to calculate the effect size of the continuous outcome measures (efficacy, satisfaction, and alliance), which is considered a valid approach [22].

    Dichotomous Outcome Measures

    The risk ratio effect size and its 95% CI was calculated for the secondary outcome attrition rate. A forest plot was created to present the effect size for each study and the overall effect size for pooled analysis and was supplemented with I² and χ² statistics.

    Additional Primary Outcome Analyses (Investigating Heterogeneity)

    Moderator Analysis

    For the primary outcome, subgroup analyses for different patient groups were performed based on (1) participant diagnosis, as specified in the eligibility criteria; (2) age; (3) length of treatment course or program; and (4) therapy type.

    The year of study publication was evaluated through a meta-regression, and the P value for the regression was computed (P≤.05 significance level).

    Sensitivity Analysis

    A sensitivity analysis was also performed to determine the robustness of the meta-analysis and included (1) sensitivity testing for only high-quality trials and (2) testing for whether the findings were sensitive to random effects or fixed effects models.

    Meta-Bias

    Publication bias was assessed visually using a funnel plot and tested statistically using the Egger test [19,23].

    Certainty of the Evidence

    The GRADE (Grading of Recommendations Assessment, Development, and Evaluation) approach as recommended by the Cochrane Collaboration was used to assess the certainty (confidence) of the evidence [19,24]. The certainty of the evidence for the primary outcome was evaluated for 5 domains and included an evaluation of the risk of bias (Rob 2), inconsistency, indirectness, imprecision, and publication bias. Each domain was graded as having a “serious,” “very serious,” or “not serious” impact on the certainty of the evidence and was downgraded with 1 level, 2 levels, or no downgrading, respectively. The GRADE approach was conducted using GRADEpro GDT software [25]. Two independent authors performed the GRADE approach (AAS and OJS), and a third author (SA) was consulted when disagreements occurred. The certainty of the evidence will be presented in the GRADE summary of the findings table.

    Ethical Considerations and Dissemination

    Ethical approval was not required for this systematic review and meta-analysis. The data sets (extraction) are deposited in the Zenodo repository (DOI:10.5281/zenodo.7339263). This study will be disseminated at scientific conferences.


    Study Selection

    This study focused on outpatient psychiatric treatment conducted via telemedicine (video) or in person for PTSD, mood disorders, and anxiety disorders. The PRISMA flow diagram shows all stages of the article identification, screening, inclusion, and exclusion processes (Figure 1). Searches generated 7414 records. Two records were identified through manual literature search and reference list scanning. After removing the duplicates and applying inclusion and exclusion criteria to the titles and abstracts, 111 reports emerged as candidates for full-text review. A total of 20 studies were included in the final review. Cohen κ coefficient indicated fair interrater reliability for the title and abstract screening process (AAS and AA: Cohen κ=0.27), whereas it was moderate for the full-text review process (AAS and SFA: Cohen κ=0.52; AAS and JS: Cohen κ=0.5).

    ‎
    Figure 1. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram across all stages of article identification.

    Study Characteristics

    Overview

    Table 1 presents the descriptive characteristics of the 20 included studies.

    Table 1. Overview of included studies.
    Study, yearSample size, nDiagnosisAge (years), mean (SD)Female, n (%)Intervention typeNumber of sessions (interval in weeks)Outcome or outcomes of interest (assessment instrument)Bias (RoB 2a) assessed for primary outcome
    Peterson et al [26], 2022120PTSDb39.95 (10.36)14 (11)Cognitive processing therapy12 (2 sessions per week)
    • Psychopathology (PCL-5c)
    • Attrition
    		High risk
    Acierno et al [27], 2021136PTSD43.4 (11.5)136 (100)Prolonged exposure14 (1)
    • Psychopathology (PCL-5)
    • Attrition
    		High risk
    Liu et al [28], 2020207PTSD48.4 (14.1)47 (23)Cognitive processing therapy12 (1)
    • Psychopathology (CAPSd)
    • Attrition
    		Some concerns
    Morland et al [29], 2020175PTSD46.5 (14.11)43 (24)Prolonged exposure6-15 (1)
    • Psychopathology (CAPS-5)
    • Attrition
    		High risk
    Watts et al [30], 2020115GADe41 (15.7)95 (83)Cognitive behavioral therapy15 (1)
    • Psychopathology (ADISf-IV)
    • Working alliance (WAI-Cg)
    • Attrition
    		High risk
    Haghnia et al [31], 201971PTSDRange: 45-60 year0 (0)Case management9 (1)
    • Attrition
    		High risk
    Acierno et al [32], 2016265PTSD45.6 (14.9)16 (6)Behavioral activation8 (1)
    • Psychopathology (PCL-Military version)
    • Attrition
    		Some concerns
    Hungerbuehler et al [33], 2016107Depression35.64 (8.33)76 (71)Case management5 (4)
    • Psychopathology (HDRSh-17)
    • Satisfaction (CSQi-8)
    • Working alliance (WAI-C)
    • Attrition
    		Some concerns
    Luxton et al [34], 2016121DepressionRange: 19-65 year22 (18)Behavioral activation8 (1)
    • Psychopathology (BDIj-II)
    • Satisfaction (CSQ-8)
    • Attrition
    		Low risk
    Maieritsch et al [35], 201690PTSD30.93 (6.05)6 (7)Cognitive processing therapyA minimum of 10 (1)
    • Psychopathology (CAPS)
    • Working alliance (WAI-C)
    • Attrition
    		Some concerns
    Egede et al [36], 2015241Depression63.9 (5.1)5 (2)Behavioral activation8 (1)
    • Psychopathology (BDI)
    • Attrition
    		Some concerns
    Morland et al [37], 2015124PTSD46.4 (11.9)100 (100)Cognitive processing therapy12 (1)
    • Psychopathology (CAPS)
    • Working alliance (WAI-C)
    • Attrition
    		Low risk
    Yuen et al [38], 201552PTSD43.98 (15.18)1 (2)Prolonged exposure8-12 (1)
    • Psychopathology (CAPS)
    • Attrition
    		Low risk
    Choi et al [39], 201485Depression65.21 (9.22)66 (78)Case management6
    • Psychopathology (HAM-Dh)
    • Attrition
    		High risk
    Stubbings et al [40], 201326Mixed Diagnosisk20 (11)15 (58)Cognitive behavioral therapy12 (1)
    • Psychopathology (DASSl subscales)
    • Working alliance (WAI-C)
    • Satisfaction (CSQ-8)
    • Attrition
    		High risk
    Chong et al [41], 2012167DepressionN/Am148 (89)Case management6 (4)
    • Psychopathology (PHQn-9)
    • Working alliance (WAI-C)
    • Attrition
    		Some concerns
    O’Reilly et al [42], 2007495Mixed DiagnosisRange: 18-65 year312 (63)Case managementUp to 4 (4)
    • Psychopathology (GSIo)
    • Satisfaction (CSQ-8)
    • Attrition
    		High risk
    De Las Cuevas et al [43], 2006140Mixed DiagnosisRange: 25-65 year93 (66)Cognitive behavioral therapy8 (3)
    • Psychopathology (SCLp-90R)
    • Attrition
    		Some concerns
    Ruskin et al [44], 2004119Depression49.7 (12.8)14 (12)Case management8 (up to 7)
    • Attrition
    		High risk
    Bishop et al [45], 200224Mixed DiagnosisRange: 18-75 year17 (71)Case management8
    • Satisfaction (CSQ-8)
    • Attrition
    		Some concerns

    aRoB 2: revised Cochrane risk-of-bias tool for randomized trials.

    bPTSD: posttraumatic stress disorder.

    cPCL: posttraumatic stress disorder checklist.

    dCAPS: Clinician-Administered Posttraumatic Stress Disorder Scale.

    eGAD: general anxiety disorder.

    fADIS: anxiety disorders interview schedule.

    gWAI-C: Working Alliance Inventory-Client version.

    hHDRS or HAM-D: Hamilton Depression Rating Scale.

    iCSQ: Client Satisfaction Questionnaire.

    jBDI: Beck Depression Inventory.

    kThe 4 studies with “mixed diagnosis” included mainly patients with PTSD and depressive disorders.

    lDASS: Depression Anxiety Stress Scales.

    mN/A: not applicable.

    nPHQ: Patient Health Questionnaire.

    oGSI: Global Severity Index.

    pSCL: symptom checklist.

    Demographics

    The number of participants included in each study ranged from 24 to 495 (mean 144, SD 101.40). Out of 20 studies, 14 (70%) studies reported mean ages ranging from 20 to 65 years (mean 44.32, SD 11.19), 5 (25%) studies reported different ranges and percentages of age, and 1 (5%) study did not provide information about participant age. Most studies included a mix of males and females (17/20, 85%); 10% (2/20) of studies consisted of only females, and 5% (1/20) of only males. The diagnoses included PTSD (9/20, 45% studies), depressive disorders (6/20, 30% studies), a mix of different diagnoses (4/20, 20% studies), and general anxiety disorder (1/20, 5% studies).

    Interventions

    A range of treatment interventions via telemedicine and in person were offered, including prolonged exposure (3/20, 15% studies), cognitive processing therapy (4/20, 20% studies), behavioral activation treatment (3/20, 15% studies), cognitive behavioral therapy (3/20, 15% studies), and case management (7/20, 35% studies). The overall number of sessions varied between 4 and 15, with a median of around 8 sessions.

    Outcomes

    This study had 4 outcomes of interest: treatment efficacy, working alliance, treatment satisfaction, and attrition rates. Attrition rates were the only outcome reported in every study (20/20, 100%), whereas efficacy was measured in 85% (17/20) of studies. Out of 20 studies, 6 (30%) studies measured treatment satisfaction using the CSQ-8, and 6 (30%) studies measured the working alliance using a version of the WAI-C.

    Treatment Efficacy (Psychopathology)

    Overview

    Data from 17 RCTs were pooled in the random-effect meta-analysis to examine the efficacy of treatment delivered via telemedicine and in person (Figure 2).

    The effect size for each study was calculated and pooled. A negative effect size favors telemedicine treatment, whereas a positive effect size favors in-person treatment. The analysis shows that the CI for the overall estimated effect size include 0 (SMD=−0.01, 95% CI −0.12 to 0.09; P=.84; I2=19%, 17 trials, n=1814). Therefore, the result indicates no statistical difference in the treatment effect between the in-person and telemedicine modalities. The estimated total heterogeneity or I2 was 19%, indicating little heterogeneity. The 3 nonpooled studies, which did not provide enough information to be included in the quantitative analysis, reported the same results, that is, no difference between in-person and telemedicine treatment regarding efficacy [31,44,45].

    ‎
    Figure 2. Forest plot (treatment efficacy) [26-30,32-43].
    Meta-Bias

    Publication bias was visually evaluated using a funnel plot (Figure 3). The funnel plot shows symmetry, indicating no risk of publication bias. The risk of publication bias was also evaluated statistically using the Egger test, which did not reveal the presence of funnel plot asymmetry (intercept=−1.718, 95% CI −3.57 to −1.818, t15=−1.818; P=.09, 17 trials, n=1814).

    ‎
    Figure 3. Funnel plot (efficacy). SMD: standardized mean difference.
    Certainty of the Evidence

    The certainty of the evidence was rated as moderate according to GRADE (Figure 4). The downgrading was due to the risk of bias in multiple trials (poor reporting of how studies were planned and conducted). Of the 17 included trials, only 3 (18%) studies were rated as having a low risk of bias, 7 (41%) studies were rated as having some concerns of bias, and 7 (41%) studies were rated as having a high risk of bias.

    ‎
    Figure 4. GRADE (Grading of Recommendations Assessment, Development, and Evaluation) summary of findings (treatment efficacy). *Evidence limited due to risk of bias.
    Additional Primary Outcome Analyses (Investigating Heterogeneity)
    Moderator Analyses

    Moderator analyses were performed on 5 moderators: 4 performed in the subgroup analyses (age, diagnosis, number of sessions, and type of treatment) and 1 performed in a meta-regression analysis (publication year). The subgroup analyses presented in Table 2 indicate no statistical differences between the aggregated subgroups for any potential moderators. Most subgroup analyses showed little to moderate heterogeneity. In general, the results of the subgroup analyses must be interpreted with caution because of the small number of trials (<10) included in each subgroup analysis.

    The meta-regression analysis included 17 trials and revealed no association between the estimated effect size and year of publication (F1,15=0.355; P=.56; I2=19.84, 17 trials, n=1814).

    Table 2. Subgroup analyses.
    ModeratorIncluded studies, nSMDa (95% CI)I2χ2 (df)Sample size, nP valuePsubgroup value
    Age.87

    		Age (range values: 18-75 years)4–0.03 (0.25 to 0.18)0.435.3 (3)635.15

    		Mean age (20-44 years)7–0.04 (–0.25 to 0.17)0.298.5 (6)522.20

    		Mean age (45-65 years)60.02 (–0.14 to 0.19)0.125.7 (5)657.34
    Diagnosis.60

    		Mixed3–0.01 (–0.25 to 0.22)0.222.6 (2)439.28

    		PTSDb80.04 (–0.12 to 0.20)0.218.9 (7)820.26

    		Depression5–0.08 (–0.33 to 0.16)0.396.5 (4)438.16
    Number of sessions.72

    		>890.00 (–0.17 to 0.18)0.3211.7 (8)824.16
    ≤88–0.04 (–0.17 to 0.10)0.087.6 (7)990.37
    Treatment.82

    		Prolonged exposure30.11 (–0.12 to 0.34)01.3 (2)304.54

    		Cognitive Processing Therapy4–0.01 (–0.34 to 0.31)0.577.0 (3)380.07

    		Behavioral activation3–0.04 (–0.28 to 0,20)0.032.1 (2)282.36

    		Cognitive Behavioral Therapy3–0.07 (–0.36 to 0.23)0.262.7 (2)270.26

    		Case Management4–0.07 (–0.29 to 0.15)0.384.8 (3)578.19

    aSMD: standardized mean difference.

    bPTSD: posttraumatic stress disorder.

    Sensitivity Analyses

    Sensitivity analyses were performed to determine the robustness of the meta-analysis. It included (1) removing low-quality studies (ie, studies rated as “some concerns” and “high concerns” assessed using RoB 2) and (2) testing whether the findings are sensitive to fixed effects models. Three studies, judged to be high-quality studies, were pooled together and yielded results supporting the robustness of the meta-analysis, that is, no difference in the efficacy between in-person and telemedicine modalities (SMD=0.00, 95% CI −0.24 to 0.25; P=.98; I2=0%, 3 trials, n=263). The overall estimated effect size was not sensitive to the fixed effects model (SMD=0.00, 95% CI −0.10 to 0.09; P=.92; I2=19%, 17 trials, n=1814), further supporting the robustness of the meta-analysis.

    Patient Satisfaction

    Patient satisfaction was assessed using CSQ-8, which was used in 6 trials. The forest plot shows the overall estimated effect size for the pooled trials (Figure 5). The analysis shows that the overall estimated effect size include 0 (MD=−0.66, 95% CI −1.60 to 0.28; P=.17; I2=44%, 6 trials, n=591). Therefore, the results indicate no statistical difference in patient satisfaction between the in-person and telemedicine treatment modalities. The estimated total heterogeneity for the analysis is 44%, indicating moderate heterogeneity.

    ‎
    Figure 5. Forest plot (patient satisfaction) [30,33,34,40,42,45].

    Working Alliance

    The working alliance, as rated by patients, was assessed using WAI-C, which was applied in 6 trials. The forest plot shows the overall estimated effect size for the pooled trials (Figure 6). The analysis shows that the overall estimated effect size include 0 (MD=0.95, 95% CI −0.47 to 2.38; P=.19; I2=75%, 6 trials, n=539). Thus, the results indicate no statistical difference in the working alliance between the in-person and telemedicine treatment modalities, and the levels of the working alliance are comparable for the 2 treatment modalities. However, the estimated total heterogeneity for the analysis is 75%, indicating substantial to considerable heterogeneity and is statistically significant (χ25=20; P=.001).

    ‎
    Figure 6. Forest plot (working alliance) [30,33,35,37,40].

    Attrition Rate

    All trials included in the meta-analysis (N=20) reported data on attrition rates. The number of attrition events for each modality was either directly extracted from the studies or calculated by subtracting the number of completers from the number of those randomized to a treatment modality. The forest plot shows the overall estimated effect size for the pooled trials assessing the attrition ratio between the modalities (Figure 7). The analysis shows that the overall estimated effect size include 1; therefore, the results indicate no statistical difference in patient attrition rates between the in-person and telemedicine treatment modalities (risk ratio 1.07, 95% CI 0.94-1.21; P=.32; I2=0%, 20 trials, n=2804). The estimated total heterogeneity for the analysis is 0%, indicating no heterogeneity. The funnel plot and the Egger test was conducted and did not reveal the presence of asymmetry, indicating no risk of publication bias. Post hoc analysis was conducted to evaluate attrition outcomes based on diagnosis (PTSD and Depression), which did not influence the overall findings.

    ‎
    Figure 7. Forest plot (attrition rate).

    Principal Findings

    The primary objective of this systematic review and meta-analysis was to evaluate whether individual psychiatric outpatient treatment for adults with PTSD, mood disorders, and anxiety disorders using telemedicine is comparable with in-person treatment. Second, the study evaluated whether patient satisfaction, working alliance using standard measures, and attrition rates were comparable between telemedicine and in-person treatment modalities. A comprehensive literature search for RCTs comparing telemedicine with in-person modalities was conducted. Using stringent eligibility criteria, 20 RCTs that met prespecified eligibility criteria were identified [14]. Overall, the study results indicate that treatment delivered through telemedicine is comparable with in-person treatment modality regarding treatment efficacy, patient satisfaction, working alliance, and attrition rate.

    The primary outcome evaluated in this systematic review was efficacy and included 17 trials assessing treatment effects between the telemedicine and in-person treatment modalities. The results did not indicate a risk of publication bias. To test the robustness of the overall finding, that is, no difference in the treatment effect between telemedicine and in-person treatment modality regarding efficacy, prespecified sensitivity analyses were conducted, which supported the overall finding. Different moderators were tested for their potential influence on efficacy and included different age groups, diagnoses, number of sessions, types of psychotherapy, and publication year. None of the evaluated moderators affected the overall findings, strengthening the evidence for the equality between telemedicine and in-person treatment on various clinical and methodological characteristics. The results of this systematic review and meta-analysis are consistent with those of previously published meta-analyses, indicating the nonsuperiority of in person to telemedicine across psychiatric diagnoses [11-13]. However, when interpreting the results for the moderators in the subgroup analysis, it is important to note that relatively few studies (<10) were included in each moderator analysis, which is a considerable limitation [46]. The meta-analysis focused on PTSD, mood disorders, and anxiety disorders in which most trials included were focused on PTSD (n=8) and depression (n=5). There were no studies included that compared the treatment effects between telemedicine and in person for patients with personality disorders, social phobia, and agoraphobia. Further studies are required to determine the efficacy of psychiatric treatment using telemedicine for these disorders.

    The secondary outcomes investigated between the 2 treatment modalities included patient satisfaction, working alliance, and attrition rates.

    Satisfaction measurement was limited to CSQ-8, resulting in 6 RCTs in the meta-analysis. The results indicate that satisfaction between treatment modalities (in person and telemedicine) is comparable, and heterogeneity is low to moderate. To our knowledge, satisfaction has only been evaluated in a single meta-analysis by Hyler et al [47], who concluded the equivalence in satisfaction between telemedicine and in-person treatment modalities regarding psychiatric assessment. However, the authors also mention limitations with the study owing to the ad hoc and untested instruments applied for measuring satisfaction and pooling the satisfaction measure for both the patients and therapists in the same analysis. This meta-analysis on satisfaction was restricted to a single validated questionnaire (CSQ-8), strengthening the evidence for equivalence in treatment satisfaction between telemedicine and in-person modalities.

    The assessment of working alliance was limited to alliance reported by patients (WAI-C), resulting in 6 RCTs in the meta-analysis. The results indicate that the working alliance between treatment modalities was comparable, although heterogeneity was substantial to considerable (I2=75%; χ25=20; P=.001), weakening the finding. Further analyses investigating heterogeneity were not possible because of the small number of included studies applying WAI-C, and this was not prespecified in the protocol. A working alliance is considered an important factor in psychotherapy outcomes, and clinicians have shown some concern that the telemedicine format of treatment may negatively impact the working alliance [48,49]. Previous research on alliance in telemedicine interventions has shown varied results but generally shows that a good therapeutic alliance can be established in telemedicine interventions and is comparable with in-person interventions [50]. Although patients consistently rate working alliance as good in telemedicine interventions, therapists have shown a tendency to rate the alliance as lower than in-person interventions. Norwood et al [3] evaluated the working alliance in a noninferiority meta-analysis and found that alliance in telemedicine is inferior to in person. Although this study combined ratings of patient and therapist alliance, the authors acknowledged that this procedure may have reduced the overall levels of alliance in the telemedicine condition. To our knowledge, this meta-analysis is the first study to examine therapeutic alliance as rated only by the patients. The comparable levels of alliance reported by patients are further supported by the similar levels of attrition and patient satisfaction rates between the telemedicine and in-person interventions found in this meta-analysis. Future studies should aim to examine how the working alliance is established over time and its implications for outcomes when treatment is delivered through telemedicine.

    The attrition rate was comparable between treatment modalities (telemedicine and in person) across diagnoses (PTSD and depression) and is consistent with the results published by Giovanetti et al [13]. The authors included 11 RCTs and found equivalent attrition rates between the telemedicine and in-person treatment modalities; however, the authors only assessed attrition rates in patients with depression. Thus, the telemedicine modality of psychiatric treatment did not appear to negatively impact sustained engagement when compared with treatment via the in-person modality.

    Study Limitations

    First, the stringent eligibility criteria regarding population, intervention, control, and outcome measures limited the total number of studies available for analysis. Second, the inadequate and poor reporting of the included studies led to only 3 studies being evaluated as high-quality based on the RoB 2 criteria. Owing to the limited number of high-quality studies, the certainty of the evidence (according to GRADE) was rated as moderate, and a high certainty of the evidence was therefore not achievable for the efficacy outcome. Third, data were nonuniformly reported in a number of the included studies. For example, age was reported as ranges (percentages) in some studies, whereas other studies reported the mean age, making it challenging to aggregate subgroups for the moderator analysis. Finally, most studies did not provide separate results for male and female participants, so it was not possible to determine whether there was a differential effect of sex.

    Implications for Clinical Practice and Research

    Rapid digitalization within mental health care is changing clinical practice. This study provides a comprehensive and up-to-date meta-analytic overview of the use of telemedicine in individual psychiatric treatment across a range of diagnoses and clinical outcomes. Evidence generated by this meta-analysis can guide clinical practice regarding which disorders can be effectively treated using individual psychiatric treatment conducted via telemedicine and which psychotherapy approaches are effective.

    Importantly, the results of this review revealed a complete absence of high-quality studies examining the efficacy of psychiatric treatment via telemedicine for personality disorders and a range of anxiety disorders. This knowledge gap must be addressed in future studies. Future studies should also examine how to identify the most suitable treatment modality (telemedicine, in person, telephone, etc) for psychiatric patients, thereby matching patient needs to the treatment mode to optimize outcomes. Using individual patient data, meta-analysis within telemedicine applications could potentially address this challenge.

    Conclusions

    In summary, the results of this meta-analysis indicate that psychiatric treatment via telemedicine for PTSD, mood disorders, and anxiety disorders were equivalent to in-person treatment in terms of treatment efficacy, satisfaction, and attrition rate. Although working alliance as rated by patients was also deemed to be comparable between the 2 modalities, heterogeneity in the analysis was substantial. Thus, there is a need for further high-quality controlled trials to fully understand the complex issue of working alliance for interventions conducted via telemedicine.

    Authors' Contributions

    AAS is the guarantor of the systematic review and wrote the first draft. OJS provided Statistical Consultation and Rating the Certainty in Evidence (Grading of Recommendations Assessment, Development, and Evaluation). JPS contributed to data extraction, risk of bias assessment, and the table overview of included studies. AA contributed to records screening. JAS and HB contributed with scientific knowledge on information technology and health care technology. KT contributed with general scientific knowledge regarding telemedicine. ES developed the idea and rationale for the systematic review. SFA designed the data extraction form, data extraction, and risk of bias assessment. All authors revised and approved the final manuscript.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist.

    PDF File (Adobe PDF File), 74 KB
    1. Chakrabarti S. Usefulness of telepsychiatry: a critical evaluation of videoconferencing-based approaches. World J Psychiatry. Sep 22, 2015;5(3):286-304. [FREE Full text] [CrossRef] [Medline]
    2. Hubley S, Lynch SB, Schneck C, Thomas M, Shore J. Review of key telepsychiatry outcomes. World J Psychiatry. Jun 22, 2016;6(2):269-282. [FREE Full text] [CrossRef] [Medline]
    3. Norwood C, Moghaddam NG, Malins S, Sabin-Farrell R. Working alliance and outcome effectiveness in videoconferencing psychotherapy: a systematic review and noninferiority meta-analysis. Clin Psychol Psychother. Nov 2018;25(6):797-808. [CrossRef] [Medline]
    4. Cowan KE, McKean AJ, Gentry MT, Hilty DM. Barriers to use of telepsychiatry: clinicians as gatekeepers. Mayo Clin Proc. Dec 2019;94(12):2510-2523. [CrossRef] [Medline]
    5. Hailey D, Ohinmaa A, Roine R. Limitations in the routine use of telepsychiatry. J Telemed Telecare. 2009;15(1):28-31. [CrossRef] [Medline]
    6. Bennett CB, Ruggero CJ, Sever AC, Yanouri L. eHealth to redress psychotherapy access barriers both new and old: a review of reviews and meta-analyses. J Psychother Integr. Jun 2020;30(2):188-207. [FREE Full text] [CrossRef]
    7. Wright JH, Caudill R. Remote treatment delivery in response to the COVID-19 pandemic. Psychother Psychosom. 2020;89(3):130-132. [FREE Full text] [CrossRef] [Medline]
    8. Wells SY, Morland LA, Wilhite ER, Grubbs KM, Rauch SA, Acierno R, et al. Delivering prolonged exposure therapy via videoconferencing during the COVID-19 pandemic: an overview of the research and special considerations for providers. J Trauma Stress. Aug 2020;33(4):380-390. [FREE Full text] [CrossRef] [Medline]
    9. Doraiswamy S, Abraham A, Mamtani R, Cheema S. Use of telehealth during the COVID-19 pandemic: scoping review. J Med Internet Res. Dec 01, 2020;22(12):e24087. [FREE Full text] [CrossRef] [Medline]
    10. Reeves JJ, Ayers JW, Longhurst CA. Telehealth in the COVID-19 era: a balancing act to avoid harm. J Med Internet Res. Feb 01, 2021;23(2):e24785. [FREE Full text] [CrossRef] [Medline]
    11. Drago A, Winding TN, Antypa N. Videoconferencing in psychiatry, a meta-analysis of assessment and treatment. Eur Psychiatry. Aug 2016;36:29-37. [CrossRef] [Medline]
    12. Batastini AB, Paprzycki P, Jones AC, MacLean N. Are videoconferenced mental and behavioral health services just as good as in-person? A meta-analysis of a fast-growing practice. Clin Psychol Rev. Feb 2021;83:101944. [CrossRef] [Medline]
    13. Giovanetti AK, Punt SE, Nelson EL, Ilardi SS. Teletherapy versus in-person psychotherapy for depression: a meta-analysis of randomized controlled trials. Telemed J E Health. Aug 2022;28(8):1077-1089. [CrossRef] [Medline]
    14. Shaker AA, Austin SF, Sørensen JA, Storebø OJ, Simonsen E. Psychiatric treatment conducted via telemedicine versus in-person consultations in mood, anxiety and personality disorders: a protocol for a systematic review and meta-analysis. BMJ Open. Sep 28, 2022;12(9):e060690. [FREE Full text] [CrossRef] [Medline]
    15. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. Mar 29, 2021;372:n71. [FREE Full text] [CrossRef] [Medline]
    16. Clarivate version: EndNote X9 type: 64 bit. EndNote. 2013. URL: https://endnote.com/downloads [accessed 2023-04-15]
    17. Veritas Health Innovation. Covidence. URL: https://www.covidence.org/ [accessed 2023-04-15]
    18. Review manager (RevMan). Cochrane Collaboration. 2020. URL: https://training.cochrane.org/online-learning/core-software/revman [accessed 2023-04-15]
    19. Higgins J, Thomas J, Chandler J, Cumpston M, Li T, Page M. Cochrane handbook for systematic reviews of interventions. Version 6.2. Cochrane. Feb 2021. URL: https://training.cochrane.org/handbook [accessed 2023-04-15]
    20. Larsen DL, Attkisson CC, Hargreaves WA, Nguyen TD. Assessment of client/patient satisfaction: development of a general scale. Eval Program Plann. 1979;2(3):197-207. [CrossRef] [Medline]
    21. Horvath AO, Greenberg LS. Development and validation of the working alliance inventory. J Couns Psychol. 1989;36(2):223-233. [FREE Full text] [CrossRef]
    22. da Costa BR, Nüesch E, Rutjes AW, Johnston BC, Reichenbach S, Trelle S, et al. Combining follow-up and change data is valid in meta-analyses of continuous outcomes: a meta-epidemiological study. J Clin Epidemiol. Aug 2013;66(8):847-855. [CrossRef] [Medline]
    23. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. Sep 13, 1997;315(7109):629-634. [FREE Full text] [CrossRef] [Medline]
    24. Schünemann H, Brożek J, Guyatt G, Oxman A. GRADE handbook for grading quality of evidence and strength of recommendations. The GRADE Working Group. URL: https://www.gradepro.org/terms/cite [accessed 2022-11-19]
    25. GDT: GRADEpro guideline development tool. GRADEpro. URL: https://www.gradepro.org/terms/cite [accessed 2022-11-19]
    26. Peterson AL, Mintz J, Moring JC, Straud CL, Young-McCaughan S, McGeary CA, et al. In-office, in-home, and telehealth cognitive processing therapy for posttraumatic stress disorder in veterans: a randomized clinical trial. BMC Psychiatry. Jan 17, 2022;22(1):41. [FREE Full text] [CrossRef] [Medline]
    27. Acierno R, Jaffe AE, Gilmore AK, Birks A, Denier C, Muzzy W, et al. A randomized clinical trial of in-person vs. home-based telemedicine delivery of prolonged exposure for PTSD in military sexual trauma survivors. J Anxiety Disord. Oct 2021;83:102461. [CrossRef] [Medline]
    28. Liu L, Thorp SR, Moreno L, Wells SY, Glassman LH, Busch AC, et al. Videoconferencing psychotherapy for veterans with PTSD: results from a randomized controlled non-inferiority trial. J Telemed Telecare. Oct 2020;26(9):507-519. [CrossRef] [Medline]
    29. Morland LA, Mackintosh MA, Glassman LH, Wells SY, Thorp SR, Rauch SA, et al. Home-based delivery of variable length prolonged exposure therapy: a comparison of clinical efficacy between service modalities. Depress Anxiety. Apr 2020;37(4):346-355. [CrossRef] [Medline]
    30. Watts S, Marchand A, Bouchard S, Gosselin P, Langlois F, Belleville G, et al. Telepsychotherapy for generalized anxiety disorder: impact on the working alliance. J Psychother Integr. Jun 2020;30(2):208-225. [FREE Full text] [CrossRef]
    31. Haghnia Y, Samad-Soltani T, Yousefi M, Sadr H, Rezaei-Hachesu P. Telepsychiatry-based care for the treatment follow-up of Iranian war veterans with post-traumatic stress disorder: a randomized controlled trial. Iran J Med Sci. Jul 2019;44(4):291-298. [FREE Full text] [CrossRef] [Medline]
    32. Acierno R, Gros DF, Ruggiero KJ, Hernandez-Tejada BM, Knapp RG, Lejuez CW, et al. Behavioral activation and therapeutic exposure for posttraumatic stress disorder: a noninferiority trial of treatment delivered in person versus home-based telehealth. Depress Anxiety. May 2016;33(5):415-423. [CrossRef] [Medline]
    33. Hungerbuehler I, Valiengo L, Loch AA, Rössler W, Gattaz WF. Home-based psychiatric outpatient care through videoconferencing for depression: a randomized controlled follow-up trial. JMIR Ment Health. Aug 03, 2016;3(3):e36. [FREE Full text] [CrossRef] [Medline]
    34. Luxton DD, Pruitt LD, Wagner A, Smolenski DJ, Jenkins-Guarnieri MA, Gahm G. Home-based telebehavioral health for U.S. military personnel and veterans with depression: a randomized controlled trial. J Consult Clin Psychol. Nov 2016;84(11):923-934. [CrossRef] [Medline]
    35. Maieritsch KP, Smith TL, Hessinger JD, Ahearn EP, Eickhoff JC, Zhao Q. Randomized controlled equivalence trial comparing videoconference and in person delivery of cognitive processing therapy for PTSD. J Telemed Telecare. Jun 2016;22(4):238-243. [CrossRef] [Medline]
    36. Egede LE, Acierno R, Knapp RG, Walker RJ, Payne EH, Frueh BC. Psychotherapy for depression in older veterans via telemedicine: effect on quality of life, satisfaction, treatment credibility, and service delivery perception. J Clin Psychiatry. Dec 2016;77(12):1704-1711. [CrossRef] [Medline]
    37. Morland LA, Mackintosh M, Rosen CS, Willis E, Resick P, Chard K, et al. Telemedicine versus in-person delivery of cognitive processing therapy for women with posttraumatic stress disorder: a randomized noninferiority trial. Depress Anxiety. Nov 2015;32(11):811-820. [CrossRef] [Medline]
    38. Yuen EK, Gros DF, Price M, Zeigler S, Tuerk PW, Foa EB, et al. Randomized controlled trial of home-based telehealth versus in-person prolonged exposure for combat-related PTSD in veterans: preliminary results. J Clin Psychol. Jun 2015;71(6):500-512. [CrossRef] [Medline]
    39. Choi NG, Hegel MT, Marti CN, Marinucci ML, Sirrianni L, Bruce ML. Telehealth problem-solving therapy for depressed low-income homebound older adults. Am J Geriatr Psychiatry. Mar 2014;22(3):263-271. [FREE Full text] [CrossRef]
    40. Stubbings DR, Rees CS, Roberts LD, Kane RT. Comparing in-person to videoconference-based cognitive behavioral therapy for mood and anxiety disorders: randomized controlled trial. J Med Internet Res. Nov 19, 2013;15(11):e258. [FREE Full text] [CrossRef] [Medline]
    41. Chong J, Moreno F. Feasibility and acceptability of clinic-based telepsychiatry for low-income Hispanic primary care patients. Telemed J E Health. May 2012;18(4):297-304. [CrossRef] [Medline]
    42. O'Reilly R, Bishop J, Maddox K, Hutchinson L, Fisman M, Takhar J. Is telepsychiatry equivalent to face-to-face psychiatry? Results from a randomized controlled equivalence trial. Psychiatr Serv. Jun 2007;58(6):836-843. [CrossRef] [Medline]
    43. De Las Cuevas C, Arredondo MT, Cabrera MF, Sulzenbacher H, Meise U. Randomized clinical trial of telepsychiatry through videoconference versus face-to-face conventional psychiatric treatment. Telemed J E Health. Jun 2006;12(3):341-350. [CrossRef] [Medline]
    44. Ruskin P, Silver-Aylaian M, Kling MA, Reed SA, Bradham DD, Hebel JR, et al. Treatment outcomes in depression: comparison of remote treatment through telepsychiatry to in-person treatment. Am J Psychiatry. Aug 2004;161(8):1471-1476. [CrossRef] [Medline]
    45. Bishop JE, O'Reilly RL, Maddox K, Hutchinson LJ. Client satisfaction in a feasibility study comparing face-to-face interviews with telepsychiatry. J Telemed Telecare. 2002;8(4):217-221. [CrossRef] [Medline]
    46. Deeks JJ, Higgins JP, Altman DG. Analysing data and undertaking meta-analyses. Cochrane Training. URL: https://training.cochrane.org/handbook/current/chapter-10 [accessed 2021-11-15]
    47. Hyler SE, Gangure DP, Batchelder ST. Can telepsychiatry replace in-person psychiatric assessments? A review and meta-analysis of comparison studies. CNS Spectr. May 2005;10(5):403-413. [CrossRef] [Medline]
    48. Rees CS, Stone S. Therapeutic alliance in face-to-face versus videoconferenced psychotherapy. Prof Psychol Res Pr. 2005;36(6):649-653. [FREE Full text] [CrossRef]
    49. Simpson SG, Reid CL. Therapeutic alliance in videoconferencing psychotherapy: a review. Aust J Rural Health. Dec 2014;22(6):280-299. [CrossRef] [Medline]
    50. Berger T. The therapeutic alliance in internet interventions: a narrative review and suggestions for future research. Psychother Res. Sep 2017;27(5):511-524. [CrossRef] [Medline]

    ‎
    CSQ-8: Client Satisfaction Questionnaire-8
    GRADE: Grading of Recommendations Assessment, Development, and Evaluation
    MD: mean difference
    MeSH: Medical Subject Headings
    PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses
    PTSD: posttraumatic stress disorder
    RCT: randomized controlled trial
    RoB 2: revised Cochrane risk-of-bias tool for randomized trials
    SMD: standardized mean difference
    WAI-C: Working Alliance Inventory-Client version

    Edited by J Torous; submitted 05.12.22; peer-reviewed by Z Niu, Y Chu; comments to author 06.03.23; revised version received 28.04.23; accepted 22.05.23; published 05.07.23.

    Copyright

    ©Ali Abbas Shaker, Stephen F Austin, Ole Jakob Storebø, Julie Perrine Schaug, Alaa Ayad, John Aasted Sørensen, Kristine Tarp, Henrik Bechmann, Erik Simonsen. Originally published in JMIR Mental Health (https://mental.jmir.org), 05.07.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on https://mental.jmir.org/, as well as this copyright and license information must be included.